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ABSTRACT 

Several  Total  Variation  (TV)  regularization  methods  have 
recently  been  proposed  to  address  denoising  under  mixed 
Gaussian  and  impulse  noise.  While  achieving  high-quality 
denoising  results,  these  new  methods  are  based  on  compli¬ 
cated  cost  functionals  that  are  difficult  to  optimize,  which 
negatively  affects  their  computational  performance. 

In  this  paper  we  propose  a  simple  cost  functional  consist¬ 
ing  of  a  TV  regularization  term  and  ^2  and  data  fidelity 
terms,  for  Gaussian  and  impulse  noise  respectively,  with  lo¬ 
cal  regularization  parameters  selected  by  an  impulse  noise 
detector.  The  computational  performance  of  the  proposed 
algorithm  greatly  exceeds  that  of  the  state  of  the  art  algo¬ 
rithms  within  the  TV  framework,  and  its  reconstruction  qual¬ 
ity  performance  is  competitive  for  high  noise  levels,  for  both 
grayscale  and  vector-valued  images. 

Index  Terms —  Image  Restoration,  Total  Variation,  Gaus¬ 
sian  noise.  Impulse  noise 

1.  INTRODUCTION 

Acquired  digital  images  are  frequently  subject  to  additive 
Gaussian  noise  [1,  Ch.  7];  if  we  then  add  the  degradation  due 
to  transmission  errors,  the  observed  image  will  be  corrupted 
by  Gaussian  with  salt-and-pepper  (constant  value  impulse) 
noise  or  random-value  impulse  noise.  Given  u*  and  b,  the 
original  and  observed  images  respectively,  then  the  mixed 
Gaussian  and  impulse  noise  model  is  summarized  by 

V  =  u  -f  77 

r(fc),  with  probability  p 
v{k),  with  probability  1  —  p 

where  v  represents  the  original  image  u*  cormpted  with  ad¬ 
ditive  Gaussian  noise  rj,  which  is  assumed  to  be  a  zero  mean 
Gaussian  random  variable  (p,,  =  0)  with  unknown  variance 
(T^,  and  h[k)  represents  the  elements  of  b.  r{k)  is  either  the 
salt-and-pepper  noise  or  the  random  value  impulse  noise;  for 
the  former  case,  r{k)  =  Cmin  or  r[k)  =  Cmax  with  probabil¬ 
ity  Pi  and  p2  respectively  (p  =  pi  -I-P2)  and  for  the  latter  case 
r[k)  is  drawn  with  a  uniform  distribution  in  [Cmin,  c^ax]- 
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A  number  of  algorithms,  some  of  which  are  based  on  TV 
regularization,  have  recently  been  proposed  for  denoising  of 
images  subject  to  the  mixed  noise  model  (1)  [2,  3,  4,  5,  6]. 
The  key  idea  of  of  the  non-TV  based  methods  is  to  use  a  two- 
phase  approach:  detect  the  outlier  pixels  before  proceeding 
with  the  filtering  phase.  For  example,  [2]  introduced  an  uni¬ 
versal  noise  removal  filter  that  first  detect  the  impulse  cor¬ 
rupted  pixels,  and  estimates  its  local  statistics  and  incorporate 
them  into  the  bilateral  filter  [7],  resulting  in  the  trilateral  fil¬ 
ter.  In  [5]  somewhat  similar  ideas  where  used  to  develop  a 
similarity  principle  which  in  turn  drives  the  weights  of  the 
“mixed  noise  filter”;  the  reconstmction  performance  (as  re¬ 
ported  in  [5])  outperforms  the  trilateral  filter. 

Within  the  TV  framework,  most  of  the  algorithms  also 
start  with  an  outlier  detection  pre-processing  phase  followed 
by  a  denoising  phase.  The  approach  in  [3]  is  based  on  two 
augmented  cost  functionals  ([3,  eqs.  (15)-(16)])  which  have 
to  be  chosen  depending  on  the  noise  characteristics;  the 
reconstruction  performance  is  competitive,  but  its  compu¬ 
tational  performance  is  extremely  poor.  In  [4]  the  compu¬ 
tational  performance  of  [3]  is  improved,  although  it  still  is 
highly  dependent  on  the  noise  level,  specially  for  high  noise 
levels;  In  [6]  an  minimization  approach  was  proposed, 
resulting  in  a  three-phase  algorithm;  reconstmction  quality 
results  and  computational  performance  are  quite  good  when 
compared  with  published  works  (and  could  be  considered 
state-of-the-art),  but  the  proposed  cost  functional  ([6,  eq. 
(5)])  is  complicated  and  has  several  regularization  parameters 
which  have  to  be  hand-picked,  and  the  use  of  a  dictionary 
learning  second  phase  affects  the  overall  computational  per¬ 
formance.  Finally  we  mention  that  [3,  4]  only  applies  to 
grayscale  images  whereas  [6]  also  applies  to  color  images. 

In  this  paper  we  propose  a  simple  cost  functional  to 
denoise  images  corrupted  with  the  mixed  noise  model.  It 
is  computationally  efficient,  and  gives  competitive  recon¬ 
struction  quality,  particularly  for  high  noise  levels,  for  both 
grayscale  and  color  images.  The  outline  of  the  paper  is  as 
follows:  in  Section  2  we  give  a  brief  description  of  the  meth¬ 
ods  used  to  develop  the  proposed  algorithm.  In  Section  3  we 
present  our  proposed  algorithm  that  is  based  on  the  Iterative 
Reweighted  Norm  (IRN)  algorithm  [8]  and  its  extensions 
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[9,  10,  1 1];  in  Section  4  we  provide  our  computational  results 
where  we  compare  our  propose  algorithm  with  [4,  6];  finally 
in  Section  5  we  give  our  concluding  remarks. 


2.  PRELIMINARIES 


The  standard  ^^-TV  regularized  solution  of  the  inverse  prob¬ 
lem  involving  the  observed  image  data  b  is  the  minimum  of 
the  functional 


r(u)  =  i 


^  A 

f 

u  —  b 

H — 
P  5 

'^{DxUn)'^  +  {DyUnY" 


n£C 


Q 

(2) 


with  C  =  {1}  (i.e.  grayscale  images)  p  =  2  and  g  =  1,  as 
described  in  [12].  The  fidelity  term  Fp{u)  =  2||u  —  b||P 
is  determined  by  the  noise  model  (p  =  2  for  the  Gaus¬ 
sian  noise  model)  and  the  regularization  term  Rq{u)  = 


{Dx^Unf  +  {DyUn)‘ 


with  g  =  1  is  the  dis¬ 


cretization  of  |Vu|,  as  originally  proposed  in  [12] 
is  also  a  valid  the  discretization  of  |Vu|  for  coupled  chan¬ 
nels).  Dx  and  Dy  represent  horizontal  and  vertical  dis¬ 
crete  derivative  operators  respectively.  It  is  assumed  that 
2-dimensional  images  are  represented  by  1  -dimensional  vec¬ 
tors;  u„  (n  e  C)  is  a  1-dimensional  (column)  or  ID  vector 
that  represents  a  2D  grayscale  image  obtained  via  any  or¬ 
dering  (although  the  most  reasonable  choices  are  row-major 
or  column-major)  of  the  image  pixels.  Furthermore,  for 
C  =  {1,2,3}  (i.e.  three-channel  color  images)  we  have 
that  u  =  [(ui)^(u2)^(u3)^]^  is  a  ID  (column)  vector  that 
represents  a  2D  color  image. 

The  original  TV  method  has  been  extended  into  a  more 
general  technique  for  inverse  problems  including  deblurring, 
blind  deconvolution  and  inpainting  [13].  It  has  also  been 
extended  to  handle  several  noise  models,  such  the  salt-and- 
pepper  noise  model,  the  non-homogeneous  Poisson  noise 
model  and  the  Speckle  (multiplicative)  noise  model  (see 
among  several  others  [14,  15,  16]  respectively). 


2.1.  IRN  approach 

The  Iteratively  Reweighted  Norm  (IRN)  approach  was  orig¬ 
inally  developed  to  restore  images  corrupted  with  Gaussian 
noise  or  salt-and-pepper  noise  for  grayscale  images  [8],  and 
then  extended  to  handle  vector-valued  (e.g.  color)  images  [9] 
as  well  as  other  noise  models  such  the  Speckle  noise  model 
[10]  and  the  non-homogeneous  Poisson  noise  model  [11].  In 
what  follows  we  provide  a  brief  description  of  the  original 
IRN  algorithm. 

The  IRN  approach  represents  the  F’  and  norms  in  (2) 
by  the  equivalent  weighted  norms  (see  [9]  for  details); 


T('=)(u)  = 
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W: 


(fc) 


1/2 


{Au  -  b) 


W, 


(fe)V2 


Du 


+c 

'(3) 


where  is  a  constant  representing  the  solution  of  the  pre¬ 
vious  iteration,  is  a  constant  value,  Ijsr  is  a  N  x  N  iden¬ 
tity  matrix,  (g)  is  the  Kronecker  product,  C  =  {l},iV=l 
(grayscale)  or  C  =  {1,  2, 3},  iV  =  3  (vector- valued)  and 

WP  =  diag  -  b))  ,  (4) 


D  =  In  (S>  [Dx^Dy^f  w‘YY'’  =  hN  ®  (5) 

=  diag  -p 


The  functions 
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(8) 


FI  >  iR 

.  fcfl  if  kl  <  iR, 

;  defined  to  avoid  numerical  problems  when  p,q  <  2  and 
-  b  or  has  zero-valued 


3.  PROPOSED  ALGORITHM 


We  will  assume  that  JV,  the  set  of  outliers  (pixels  corrupted 
with  impulse  noise)  is  known.  For  the  scope  of  this  paper, 
the  set  Af  will  be  estimated  either  (i)  via  the  adaptive  median 
filter  [17]  for  the  salt-and-pepper  case,  or  (ii)  via  the  direc¬ 
tional  weighted  median  filter  [18]  for  the  random  value  im¬ 
pulse  noise  case.  Using  set  Af  as  given  information,  we  pro¬ 
pose  the  cost  functional  defined  by 


1 

1 

2 

A-(u-b) 

1  2Ag 

A-(u-b) 

+  Rq{u),  (9) 
2 

for  denoising  images  corrupted  with  mixed  noise,  where  A  = 
diag(I[ug7\/]),  A  =  1  —  A,  I[  ]  is  the  indicator  function,  1 
represents  an  identity  matrix  (with  the  proper  dimensions), 
Rq{u)  with  g  =  1  is  defined  as  in  (2)  and  Xq  and  Ag  are  the 
regularization  parameters  used  on  the  outliers  and  Gaussian 
corrupted  pixels  respectively. 

Defining  Aq  =  diag(l/Ao)  *  A  and  similarly  Ag  = 
diag(l/ Ag)  *  A,  then  (9)  can  be  written  as 


T(u) 


Aq  •  (u-b) 


AG-(u-b) 


2 


+  7?g(u).  (10) 
2 


Using  the  same  idea  as  for  the  IRN  algorithm  (see  Section 
2.1),  we  can  replace  the  ^^-norm  term  in  (10)  by  a  prop¬ 
erly  weighted  f^-norm  and  combine  the  two  resulting  f^-norm 
terms  into  one; 


r(U(u) 


1 

2 


W, 
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+  (11) 
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where  Wp  =  Ap  +  Aq,  and  is  a  diagonal 

weighting  matrix  defined  as  in  (4);  here  we  stress  that  Kq  and 
Ag  induce  a  partition  on  the  set  of  pixels.  It  is  interesting  to 
notice  that  (11)  resembles  the  original  TV  problem,  except  in 

(M-1/2 

the  form  of  the  weighting  matrix  Wp  .  Furthermore,  for 
a  given  weighting  matrix,  (11)  can  be  easily  solved  since  the 
regularization  term  can  also  be  approximated  by  a  quadratic 
term,  as  for  the  IRN  algorithm.  The  final  solution  is  computed 
iteratively. 

4.  EXPERIMENTAL  RESULTS 

We  compare  our  propose  algorithm  with  two  state  of  the  art 
methods:  CCN  (Cai-Chan-Nikolova,  [4])  and  XZYN  (Xia- 
Zeng-Yu-and-Ng,  [6]).  We  use  the  peak  signal-to-noise  ratio 
PSNR  =  101og]^Q  one  of  the  reconstruction 

quality  metrics  to  match  the  results  presented  in  [4, 6];  we  also 
provide  the  SNR  and  SSIM  [19]  metrics  whenever  possible. 

Due  to  space  constraints  we  only  present  results  for  the 
grayscale,  512  x  512  pixel,  Lena  image  for  Gaussian  with 
salt-and-pepper  noise  case  and  for  the  Gaussian  with  random 
value  impulse  noise  case,  although  results  for  several  other 
images  and  cases  may  also  be  generated  using  the  the  NUMI- 
PAD  (v.  0.30)  distribution  [20],  an  implementation  of  IRN 
and  related  algorithms.  All  simulations  have  been  carried 
out  using  the  above  mentioned  Matlab-only  (version  R2009b) 
code  on  a  1.73GHz  Intel  core  i7  CPU  laptop  (L2:  6144K, 
RAM;  6G);  this  software/hardware  setup  is  comparable  with 
that  used  in  [6],  allowing  us  to  present  rough  comparison  of 
our  algorithm  with  those  presented  in  [6]. 


Image 

Noise 

SNR 

PSNR 

SSIM 

Time  (s.) 

<7 

p 

Ours 

[6]^*^ 

[4]i*t 

Ours 

Ours 

[6]^*^ 

[4]  to 

Ours 

Lena 

5 

255 

0.3 

20.11 

36.20 

34.15 

34.64 

0.89 

338 

93 

34.5 

0.5 

18.21 

33.93 

32.30 

32.75 

0.87 

- 

- 

34.4 

0.7 

15.66 

30.76 

29.73 

30.20 

0.83 

247 

119 

26.9 

10 

255 

0.3 

17.82 

33.19 

31.33 

32.36 

0.83 

- 

- 

36.4 

0.5 

16.61 

31.51 

29.88 

31.14 

0.81 

- 

- 

35.2 

0.7 

14.54 

28.98 

27.75 

29.07 

0.78 

- 

- 

30.1 

15 

255 

0.3 

16.03 

31.49 

29.67 

30.56 

0.76 

215 

143 

32.8 

0.5 

15.27 

29.95 

28.42 

29.80 

0.75 

- 

- 

38.8 

0.7 

13.80 

27.53 

26,46 

28.33 

0.74 

176 

248 

41.8 

Table  1.  Experimental  results  for  the  Gaussian  (a)  with  salt- 
and-pepper  (p)  noise  case  for  the  Lena  (512  x  512  grayscale) 
image.  Results  taken  from  [6,  Tables  7  and  10]. 


In  Tables  1  and  2  we  summarize  the  results  for  the  Gaus¬ 
sian  with  salt-and-pepper  and  Gaussian  with  random  value 
impulse  noise  cases  respectively.  The  computational  perfor¬ 
mance  of  our  proposed  algorithm  is  almost  independent  of 
the  noise  level  and/or  mixture  kind;  this  is  not  the  case  for  the 
CNN  and  XZYN  algorithms.  Moreover  our  computational 
performance  is  twice  as  fast  as  that  of  CNN,  and  an  order 


of  magnitude  faster  than  XZYN  for  several  cases.  However, 
for  the  Gaussian  with  salt-and-pepper  noise  case  our  recon¬ 
struction  quality  results  are,  on  average,  about  IdB  (PSNR) 
lower  than  XZYN  (even  though  they  are  slightly  better  than 
CCN);  similarly  for  the  Gaussian  with  random  value  impulse 
noise  case  our  reconstruction  quality  results  are  on  average  1 
dB  and  2  dB  (PSNR)  below  of  those  reported  by  CCN  and 
XZYN  respectively. 

Since  our  proposed  algorithm  can  be  applied  to  vector¬ 
valued  images  we  present  a  color  reconstruction  example:  in 
Figure  1  we  show  two  corrupted  Lena  (512  x  512)  images: 
Gaussian  {a  =  15/255)  with  salt-and-pepper  (p  =  0.7)  noise 
and  Gaussian  (a  =  15/255)  with  random-value  impulse  (p  = 
0.3)  noise  and  their  restored  versions. 
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p 
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[6]^*^ 
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15.31 

30.85 

29.34 

29.84 

0.73 

35.8 
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28.78 

28.43 

0.67 

33.5 
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12.35 

29.11 

28.20 

26.88 

0.60 

35.1 

Table  2.  Experimental  results  for  the  Gaussian  (a)  with  ran¬ 
dom  value  impulse  (p)  noise  case  for  the  Lena  (512  x  512 
grayscale)  image.  Results  taken  from  [6,  Table  8]. 

5.  CONCLUSIONS 

We  have  proposed  a  simple  yet  effective  TV  cost  functional 
for  image  restoration  under  mixed  Gaussian  and  impulse 
noise.  The  computational  performance  of  the  proposed  al¬ 
gorithm  greatly  exceeds  that  of  the  state  of  the  art  algo¬ 
rithms  [4,  6],  being  an  order  of  magnitude  faster  for  several 
cases.  There  is,  however,  a  trade-off  to  be  made  for  the 
superior  computational  performance;  for  the  Gaussian  with 
salt-and-pepper  noise  case  our  reconstruction  quality  results 
are  slightly  better  than  [4],  but  slightly  worse  than  [6],  and 
for  the  Gaussian  with  random  value  impulse  noise  case  our 
reconstruction  quality  results  are  slightly  worse  than  those 
reported  by  both  [4]  and  [6] . 

As  part  of  our  on-going  work,  we  highlight  that  our  re¬ 
construction  quality  results  could  be  improved  by  means  of  a 
spatially  (and  automatic)  adaptation  scheme  for  the  regular¬ 
ization  parameters,  such  the  one  used  in  [1 1]. 
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Fig.  1.  Color  Lena  (512  x  512)  corrupted  with  (a)  Gaus¬ 
sian  (cr  =  15/255)  with  salt-and-pepper  {p  =  0.7)  noise, 
PSNR=6.70  dB  and  with  (b)  Gaussian  (cr  =  15/255)  with 
random-value  impulse  (p  =  0.3)  noise,  PSNR=13.63  dB;  (c) 
and  (d)  are  the  denoised  version  of  (a)  and  (b)  with  PSNR 
equal  to  27.82  dB  and  26.77  dB  respectively. 
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